System And Method For Accessing Electronic Data Via An Image Search Engine

ABSTRACT

The present invention provides a system and method for accessing electronic data through entry of images as queries in search engine. The system uses various image capturing devices and communication devices to capture images and enter them into image database. Image recognition techniques encode images in a computer readable format. The processed image is then entered for comparison into at least one database populated with images and associated information. Once the newly captured image is matched with an image in the database, the information linked with that image is returned to the user.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a continuation application of U.S. patent application Ser. No.14/083,864, filed Nov. 19, 2013, which is a continuation application ofU.S. patent application Ser. No. 13/939,569, filed Jul. 11, 2013, whichis a continuation application of U.S. patent application Ser. No.13/237,849, filed Sep. 20, 2011 and issued as U.S. Pat. No. 8,510,337 onAug. 13, 2013, which is a continuation application of U.S. patentapplication Ser. No. 12/799,532, filed Apr. 27, 2010 and issued as U.S.Pat. No. 8,024,359 on Sep. 20, 2011, which is a continuation applicationof U.S. patent application Ser. No. 11/101,716, filed Apr. 8, 2005 andissued as U.S. Pat. No. 7,765,231 on Jul. 27, 2010, the entire contentsof the entire chain of applications is herein incorporated by reference.

FIELD OF THE INVENTION

The invention relates to the field of accessing and retrievingelectronic data. The system and method utilize an image acquisitiondevice and a communication device to acquire and enter an image as aquery in a database. Image recognition techniques then find relatedinformation in the database and return that information to the user.

BACKGROUND OF THE INVENTION

The Internet began as a simple database of limited textual information,and quickly transformed into an extensive database of images, text, andaudio information. It would take several lifetimes to hunt for variouskinds of information throughout the Internet and USENET news groups,and, all the while, the number of files would be expanding faster thananyone's ability to peruse them.

Search engines were devised to manage the hunt. Search engines areprograms that search the Internet for documents that contain specifiedkeywords and return a list of documents which contain those keywords.These engines run programs called “spiders” that continuously explorethe Internet and, often, USENET news groups, they index the informationon websites that the spiders encounter. Indexing forms a vast databaseof website addresses that are associated with key words that have beenfound on the websites themselves.

Search engines such as Yahoo, Google, MSN, and International BusinessMachines'CLEVER require the user to enter at least one key term or queryinto a text field. Keywords, phrases, phrases in quotes, and Booleanqueries are matched to various sites on the Internet, and when the queryis complete a list of these sites is displayed for the user's review.

Although the most widely used search engines have a category thatenables them to access images, none of them allows an image to beentered as a query or search entity. All known engines require that theuser enter a text query, and the search hits files that display imagesthat are associated with the entered text query. If a person sees animage and wishes to access online information about it, he or she willhave to search for it using a text query. The user cannot use the imageitself as a query. If the user cannot put his or her search request intowords, he or she will not be able to conduct a search in a standardonline search engine.

Several innovators are working to solve this need. Hewlett-Packard, forexample, has developed a method of indexing an image that is based oninformation derived from a global positioning system (GPS). The systemobtains an image along with its location, and indexes images accordingto their location. Such systems are useful in organizing album datasince some digital cameras can acquire GPS data and correlate it withcaptured imagery. However, searching is limited to images that have asignificant correlation with a given location.

A search engine developed by Xerox Corporation incorporates amulti-modal browsing and clustering system to retrieve image data. Thesystem seeks similarities between images not only in textual references,but also in other associated information such as in-links, out-links,image characteristics, text genre, and the like. However, this engine islimited to specific image types which have defined colors, contain text,and have other visual identifiers. In short, the Xerox engine requiresthe images to have such specific characteristics, it limits the system'sutility and viability as an all purpose search engine.

Some attempts have been made to extract information from databases usingimages themselves as search entities rather than keywords related to theimages. These systems can translate, provide information about, orinterpret objects contained in an image. These systems generally work asfollows. An input device extracts the object of interest from itsbackground. The object is compared with objects stored in apre-populated database to find a match. Finally, the system retrievesinformation in the database about the object and permits it to bedisplayed to the user. However, the system is limited to imagescontaining extractable, defined objects, such as fruits, articles,animals, or any object which is easily outlined. However many imagesrequire identification as a whole entity, such as an image of ageographic locations or a piece of artwork. As a result, this method haslimited applicability.

Complex images with a myriad of superfluous objects are easier toidentify using methods such as pixel analysis. Using this method, adatabase is populated with primitive, weighted vectors of images thatfacilitate the image processing. The inputted images are compared andmatched through specific vectors that define them. Therefore, thereremains a clear need for a system capable of capturing images,converting those images into computer readable formats, using theprocessed images as search queries in a search engine, comparing theimages to images stored in the database, and, upon finding a match,displaying information associated with the image to a user of thesystem.

SUMMARY OF THE INVENTION

The present invention allows a user to extract information about anobject, organism, or scenario of interest by acquiring its image andinputting that image into a search engine. The search engine canrecognize the image and extract related information in the form ofelectronic data. Using this system, user can extract information aboutvirtually anything, ranging from profiles of people of interest tohistoric information about a monument, or information about a piece ofartwork.

One object of the invention is the creation of a system which utilizesentry of an image as a search query or entity into a search engine.

Another object of the invention is the creation of a comprehensiveregistry of images, such as photographs, drawings, video clips, andholograms, which are associated with electronic data and serve as auniversal image database that is available for matching images enteredas search queries.

Another object of the invention is to provide the user of the systemwith the capacity to add information pertaining to an image to thedatabase.

Another object of the invention is the creation of a system whichutilizes pixel analysis as a means of comparing images entered asqueries with images in the database in order to find a best-fit match.

A further object of the invention is the creation of a system whichutilizes entry of an image along with alphanumeric characters to narrowthe search. Boolean expressions (A AND, NOT, and OR B) can link imageswith text as a means of narrowing the search. Similarly, a plurality ofimages can be used in Boolean expressions.

A further object of the invention is the creation of a system whichutilizes entry of geographical coordinates in addition to the image inorder to narrow the search. These coordinates can be entered by means ofGPS, triangulation of cellular telephone towers, or the like.

Yet a further object of the invention is the creation of a system whichutilizes entry of time and date of image capture along with the image inorder to narrow the search.

An additional object of the invention is the creation of a system whichutilizes entry of video clips or an image with a spoken word using VoiceRecognition Technology (VRT) or a conventional keyboard to furthernarrow the search.

Another object of the invention is the creation of a system whichutilizes the use of Optical Character Recognition (OCR) technology toread and interpret text associated with captured images, such that thetext is entered as a search term accompanied by such images to narrowthe search.

Another object of the invention is the creation of a system whichenables advertisers or marketers to preplan response to the entry ofimages of advertisements by providing images of said advertisements orthe products seen within to those who update the search engine, andlinks to relevant products, services, discounts, and the like.

Other objects of the invention are obtaining more information aboutproducts and services and, if desired, purchasing or leasing them. Thisobject is enabled by the user's capturing of an image of a product orpart of a product, the entry of said image as a search query, and theprovision of links to commercial Web sites by those who update thesearch engine.

Another object of the invention is the creation of a system to aideducation. In this embodiment, the user captures an image and obtainsinformation about the subject of the image from online educationalsources such as books, encyclopedias, dictionaries, translators, and thelike.

Another object of the invention is the creation of a system whichenables the user to communicate with at least one person. In thisembodiment, the user captures an image of a person of interest andobtains contact and other information posted online by or about theperson of interest.

A further object of the invention is the creation of a system which canact as a travel guide, which gives the user the capacity to capture animage and obtain information such as location, translation, historicdescription, current news, nearby attractions, where to stay, where toeat, transportation, current currency exchange, and the like.

In accordance with one embodiment the present invention comprises asystem for accessing electronic data by providing an image comprising:(i) a means for capturing an image, (ii) a means for transmitting saidimage to a database wherein the database comprises: a. a means toreceive said image, b. a means to access electronic data associated withsaid image, and c. a means to transmit said data to a display unit.

In accordance with another embodiment the present invention comprises amethod of extracting electronic data from a database by providing animage captured by capturing means comprising: (i) providing computercoded images stored on the database and further linked to electronicdata, (ii) entering captured image, (iii) performing image recognitionfunctions to computer code said captured image, (iv) matching saidcomputer coded image to said computer coded images stored on thedatabase, (v) linking said captured image to said matched linkedelectronic data, and (vi) presenting said electronic data on a displayunit.

BRIEF DESCRIPTION OF THE DRAWINGS

A further understanding of the present invention can be obtained byreference to embodiments set forth in the illustrations of theaccompanying drawings. Although the illustrated embodiments are merelyexemplary of systems for carrying out the present invention, both theorganization and method of operation of the invention, in general,together with further objectives and advantages thereof, may be moreeasily understood by reference to the drawings and the followingdescription. The drawings are not intended to limit the scope of thisinvention, which is set forth with particularity in the claims asappended or as subsequently amended, but merely to clarify and exemplifythe invention.

FIG. 1A depicts an illustration of the interaction of the majorcomponents of an image database, network, and transmission device inaccordance with the present invention.

FIG. 1B depicts a flow diagram illustrating the methods and possibleorder of component interaction the components of FIG. 1A.

FIG. 2A depicts image outputs after applying various filters of imageprocessing to differentiate an object within an image in accordance withthe present invention.

FIG. 2B depicts a flow diagram of the process steps applied to images ofFIG. 2A.

FIG. 3 depicts a screen shot of an exemplary interface of the searchengine in accordance with the present invention.

FIG. 4 depicts a screen shot of an exemplary interface of the searchengine to search images in addition to a text query via Booleanparameters in accordance with the present invention.

FIG. 5 depicts a screen shot of an exemplary interface of the searchengine to search images via an search categories in accordance with thepresent invention.

FIG. 6 depicts a flow diagram illustrating a process wherein an image iscaptured and processed in order to extract information about the imagein accordance with the present invention.

FIG. 7 depicts a flow diagram illustrating a process wherein a wirelesstransmitting device is utilized to transmit information between thecommunication device and the database in accordance with the presentinvention.

FIG. 8 depicts a flow diagram illustrating a process wherein additionalconstricting parameters such as GPS, date, and time can be used tofurther narrow and expedite the search in accordance with the presentinvention.

FIG. 9 depicts a practical use of the system of the present invention inthe commercial context, wherein a user utilizes a camera enabled PDA tocapture an image of a product and acquire purchasing information via themethod of the present invention.

FIG. 10 depicts a practical use of the system of the present inventionin the advertisement context, wherein a user utilizes a scanning deviceand a computer to scan an advertisement from a magazine and accessfurther information about the subject of the advertisement via a searchof the advertisement image or images using the method of the presentinvention.

FIG. 11 depicts a use of the system of the present invention in thecommunication context for purpose of facilitating personal contact toone or plurality of personal parties, wherein a user uses a cameraenabled phone to capture an image of a person and obtain informationabout that person of interest via the method of the present invention.

FIG. 12 depicts a use of the system of the present invention in theeducation context, wherein a user uses a video camera and a computer inorder to acquire educational information about an object depicted in acaptured video via the method of the present invention.

FIG. 13 depicts a use of the system of the present invention in thetourist context, wherein a tourist uses a GPS and web-enabled digitalcamera to capture an image and to acquire information about that imagevia the method of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

Detailed illustrative embodiments of the present invention are disclosedherein. However, techniques, systems and operating structures inaccordance with the present invention may be embodied in a wide varietyof forms and modes, some of which may be quite different form those inthe disclosed embodiment. Consequently, the specific structural andfunctional details disclosed herein are merely representative, yet inthat regard, they are deemed to afford the best embodiment for purposesof disclosure and to provide a basis for the claims herein that definethe scope of the present invention. The following presents a detaileddescription of a preferred embodiment of the present invention.

The present invention provides a system capable of capturing images,entering the images into the search engine, extracting informationassociated with the images, and presenting the information to a user.Image capturing devices 100 capture the image and then transfer theimage to communication devices 101 having transmitting and receivingmeans capable of communicating with database 103 through network 102, asshown in FIG. 1A. Wherein the transmitting and receiving means are anymeans capable of transmitting and receiving electronic signals. Theimages can be captured from a visual entity (object, people; animal,places, or anything capable of being captured by an image); entered froma printed material (photograph, book, magazine, poster, identificationcard, credit card, bank card, passport, advertisement, or any otherprinted media); copied from an electronic display unit (computermonitor, hand-held device screen, or any other similar device); capturedfrom projected visual information (still image, film, video clip,streaming hologram, etc.), or any other means known for capturingimages.

Network 102 can be of any type, including but not limited to a networkthat is wired, wireless, GSM, ISDN, Ethernet, CATV, Wi-Fi, LAN,Bluetooth, or the like. Likewise, the capturing apparatus can be anydevice capable of transferring a real-time visual entity into adigitalized image such as, but not limited to, digital/analog cameras,video cameras, scanners, hand-held scanners, camera-enabled cellulartelephones, camera-enabled PDA's, or the like. The communication devicecan be any device or combination of devices having communicationfunctions and displaying means such as, but not limited to, a hand-helddevice, cellular telephone, hybrid cellular telephone/PDA device, PDA,remote server, RFID device, Internet accessible camera, personalcomputer, laptop computer, pocket computer, hybrid electronic device, orthe like. The image-capturing device can be connected to a communicationdevice through a hard-wired data link, wireless data link, or any othertype of connection. Many image-capturing devices and communicationdevices are integrated into one unit, or can be integrated into oneunit, such that any communication device can have image capturingcapabilities and vice versa. The combination of the plurality ofimage-capturing devices and the plurality of communication devices willbe referred to as CI devices (Communicable-Imaging Devices) hereinafterdue to the difficulties with making definite distinctions between thesedevices.

A possible method of component interaction and the associated processesare depicted in FIG. 1B wherein the image-capturing device 100 capturesan image of an object, shown in process 110, which is received by thecommunication device 101, as shown in process 111. Communication device101 processes the image, as shown in 112, by storing the image,converting the image to a desired data-type, and/or obtaining andindexing additional information about the image. Communication device101 then transmits the image to the search engine's database 103 via anetwork 102 in process 113. After receiving the image in process 114,the processing means associated with database 103 performs imagerecognition functions of process 115 and compares the image with imagesstored within database 103 in process 116. After finding a match,database 103 extracts information associated with the image, as inprocess 117, and further transmits the information back to communicationdevice 101 thru network 102, as shown in process 118. Communicationdevice 101 receives the information, as shown in process 119, anddisplays the information on its display screen or an associated displaydevice of process 120.

The disclosed system utilizes image recognition technology to define animage and retrieve information about it from a large database. Manytechniques can be used for image recognition as it has been an emergingfield since the mid-1900s. The most widely-used approach for imagerecognition is object extraction, as shown in FIGS. 2A and 2B. Image 201contains both a background 210 and object 211 located in the foreground.The first step to extract the object 211 is to remove as much “noise”from the image as possible as shown in process 230. Noise can take manyforms such as vibrations from movement, particles in the air, or thelike. When these disruptions occur, they create discrepancies within animage. Though slight and not apparent to the naked eye, the noise cancause difficulties when applying mathematical (algorithmic) propertiesto an image. For example, if a person were to attempt to trace an imagethat had a defined outline, the process would be easier than attemptingto trace an image that had a fuzzy and discontinuous outline. Therefore,the more noise and less resolution an image has, the more difficult itwill be to interpret it and match it to another image. Removal of noiseis also known as noise filtering which can be seen in drasticproportions from image 201 to image 202.

Then the image 202 can be segmented in process 231 into contiguousregions where the result is seen on the segmented image 203. The nextstep in the imaging process is to filter image 203, or perform low-levelextraction in process 232, in order to completely define object 211 fromthe background 210. Once extracted and enhanced the object's lines 220are located in image 204. Following, vectors are assigned to theextracted lines and the image is stored in process 233 as a series ofvectors (matrices) that are compressed and quantized to a finite amount,which often causes loss of data and, consequently, resolution, when andif the image is later viewed. It is contemplated that the order of imageprocessing steps (e.g. noise filtering and segmentation), the numbereach step is performed, and the addition of further processing steps canvary with each application without departing from the spirit of thepresent invention.

After the object's lines 220 are defined, stored, and compressed, themathematical representations are compared to other mathematicalrepresentations of images in a database. These mathematicalrepresentations might differ slightly due to this loss of additionalinformation during processing. Therefore, when compared in a database,the information returned to the user will most likely need to contain aplurality of best-fit matches. This process of feature extraction andcomparison is called the Digital Elevation Model (DEM) for imageregistration.

In addition to the improvements being made to the current processes ofpattern recognition, image recognition, and other types of computervision, new methods are being developed to troubleshoot problematicareas of the pre-existing ones. For instance, there are manners ofextracting image data by texture, color, neural networks, location,background objects, and the like. However, these areas still requireimprovement for reliability. Nevertheless, the present inventionenvisions future applications for potential use of these newtechnologies as the image-recognition process in this invention.

The information associated with the images varies with different systemapplications. The source of information can comprise a single serviceprovider site, combination or network of sites, or the entire universeof available information on the Internet. In a single site and a singleapplication, each image or a group of images is linked to presetinformation. Essentially, each image or a group of images can have awebpage associated with it. For example, a user enters image 301 tosearch engine 300, shown in FIG. 3. The image can be entered into searchfield 302 in variety of possible ways, such as, but not limited to,cutting and pasting the image, uploading the image file, typing the pathlocation to the search engine, and the like. As the user enters theimage and initiates a search by pressing button 303, the systemidentifies the image and directs the CI device to the webpage associatedwith the image information. Additionally, these sites may require asubscription to the service and/or charge the user per each serviceusage. Since the CI device requires some communication subscription, theservices can be charged to the existing communication subscription aswell. For example, if a cellular telephone is used, the user may receivea charge on his or her cellular telephone service provider bill.However, the system might also be financially supported by sponsors'links.

If multiple sites of information are used, the images stored in thedatabase can be indexed with text identifiers or the like, such as animage title, titles, or names of objects in the image. If the userenters image 301 to search engine 300 of FIG. 3, the captured image ismatched with a stored image and associated with the indexed informationabout that stored image. This information then can be used to search theWorld Wide Web, USENET newsgroups, and other sources, to retrieveadditional desired information about the image. As well, otherrestrictions can be enforced. For example, the system might allow onlycertain websites to be searched or might prevent some websites frombeing searched. As a result, desired privacy can be protected.

When searching a large database, many matches can be found for asingular image, resulting in an excessive number of results.Consequently, the user could be presented with more source ofinformation than he or she needs. To narrow the field of the search, theuser can specify particular information she desires within the scope ofsystem application through the use of Boolean expressions as illustratedin FIG. 4. The user enters image 301 into search engine 300, chooses toadd a Boolean expression via pull down menu 400, and enters inquiryinformation into field 401, narrowing the field of the search. Booleanexpressions, such as AND, OR, NOT, and the like, can be chosen from thepull down menu or typed into the search engine manually. Inquiryinformation inputted into field 401 can be anything associated with theimage, anything the user wishes to discover about the image or anythingthe user wishes to know about in conjunction with the image. The systemcan identify captured image 301, extract information associated with theimage, and further perform a search utilizing the information associatedwith the inquiry information from field 401. In an alternative method tonarrow the search, the system can first search for images associatedwith inquiry information from field 401 then use the found group ofimages and match them to the captured image 301, and extract theinformation regarding the image. Additionally, the search engine 300could function such as disregarding the Boolean inputting field 400 andusing a default Boolean parameter to search the database. In such case,it is preferred and common in practice to use the AND parameter. Also,the utilization of OCR technology to achieve a more automated system ispossible.

The system can transform images with embedded text into key words andenter those key words as search terms for the search engine, furthershaping the extent and nature of the search. Alternatively, a series ofalphanumeric characters, such as key words, is generated and entered bya user to further clarify and narrow the search. For example, a travelercan take a photo of the window of a restaurant, capturing parts of amenu, parts of the window display, or the name of the restaurant. Whenthe photo is entered into a search engine, it returns informationpertaining to reviews, decor, value, history, or the like.

When applying the system to multiple applications, the database searchcan be arranged into categories as shown in FIG. 5. The user entersimage 301 into search engine 300, and specifies the type of imagecaptured or the type of information she desires to extract by choosingfrom category list 501. For example, different types of searches can beperformed based on what the user wants to know. For example, if theimage is of a building and the building is historic, and historicalinformation is desired, the search can be limited to historic sites. Ifthe building is a restaurant and the user desires information about it,the search may be limited to commercial dining sites. These sites mightprovide restaurant hours, proper attire, type of food, or a view of theentire food menu. The CI device is programmed to provide the user with amenu 500 in which available categories are chosen from category list501. In another embodiment, the user also may key or type a category ofinquiry, or enter it by means of VRT.

A system of the present invention might comprise a CI device connectedto a network where the process of operation is shown in FIG. 6. The CIdevice captures an image and emits an inquiry signal containing theimage in process 600. The database receives the inquiry signal through anetwork, performs image recognition, acquires information associatedwith the image, and emits a response signal containing the acquiredinformation in process 601. Finally, the CI device receives the responsesignal containing the information and displays the information as shownin process 602.

If the CI device is wireless, a wireless transmitting device, such as aremote tower, is used to transfer the information from the CI device toa network, the process of its operation is shown in FIG. 7. The CIdevice captures the image and emits an inquiry signal as shown inprocess 700. The wireless transmitting device then receives the signaland transfers it to a database through a network in process 701. Thedatabase then performs image recognition, acquires informationassociated with the image, and sends the information back to thewireless transmitting device in process 702 to be transferred back tothe CI device in process 703. The CI device receives the information anddisplays it for the user on any number of wireless CI devices such ashand-held devices, cellular telephones, PDA's, laptop computers, or thelike.

FIG. 8 shows a process of operation where a GPS-equipped CI device canadditionally record the date and time of image capture. The CI devicefirst captures an image, then records the time and date of imagecapture, and finally emits an inquiry signal with a data-packetcontaining the acquired information in process 800. A wirelesstransmitting device receives the inquiry signal and transfers it to GPSsatellite in process 801. When a satellite receives the signal, the CIdevice coordinates are calculated and indexed into the data-packet asshown in process 802. The wireless transmitting device receives theindexed data packet and transfers it to the database for analysis inprocess 803. The processing means associated with the database performsimage recognition functions and acquires information associated with theimage and any additional information provided in process 804. The CIdevice receives the information through wireless transmitting device inprocess 805 and displays it on display screen in process 806 or anassociated display unit.

The present invention has an important applicability in the commercialsphere. The ability to capture a product image or an image related to aproduct, acquire information about it, and purchase it by means of theCI device is desirable. The CI device might capture an image of aproduct in a store or of a product of interest in the possession ofanother party. Alternatively, the user can capture a product image fromanother image, such as a pamphlet, TV commercial, monitor of a computer,screen of a hand-held device, magazine, newspaper, product label,poster, or the like. Furthermore, the system enables a user to capturean image of any person, place, or thing, to receive information aboutthe object, and to take a subsequent action such as making a purchase,leasing a product, arranging financing, or arranging delivery or pick-upof the product.

When capturing an image of product labels, various printed indicatorscan be useful for fast and accurate image recognition. Barcodes, serialnumbers, model numbers, or any other identifying parameter can help toidentify the product, since they are unique. Examples of commercialapplications include, but are not limited to, real estate, retailstores, entertainment, and other such venues.

FIG. 9 shows an embodiment of the invention to acquire information aboutproduct 901 on display in store window 900. The user operates hand-heldCI device 903, such as a hybrid PDA/cellular telephone with cameraattachment 904, in order to capture image 902 of product 901. CI device903 emits an inquiry signal to be received by wireless transmittingdevice 905 and transferred to network 906 that contains a database.Database 907 receives image 902, performs image recognition, andaccesses electronic information associated with that image. CI device903 receives the information that was accessed and displays it.

The user of the system may be interested in the product, but not havethe inclination to review the information about it as soon as it isretrieved due to time, money, and/or availability constraints. Toaccommodate for this, the system allows the user to capture an image ofthe product and store it for later use. Additionally, one might capturea desired product with an unwanted detail, such as color, size, or thelike, and use the system to identify the product. Upon identification ofthe product, the user can then access additional information aboutavailability of variations of the product and locations to purchase it.

The product information associated with product 901 might consist of,but is not limited to, product description 911, pricing, store locationsand availability, online purchase capabilities, purchase statistics,information about related products, and the like. Additionally, theinformation might consist of links to a plurality of retail store sites912, product manufacturers, online stores 913, online auction sites, andthe like. After reviewing the product information, the user is able topurchase the product using the acquired information. Alternatively,after capturing an image and instead of acquiring product information,the CI device is directed to an order placement site wherein the usercan readily place an order.

Preferably, each user of the system has a personal profile such that thesystem can acquire information according to the user's criteria. Theprofile might consist of price limits, residency, taste, sizes, and thelike. In another embodiment, providing the system with the residence orworkplaces of a user allows the search engine to extract proximate storelocations. Moreover, the user might enter a current location, or thesystem might have positioning capabilities such as GPS to find proximatelocations to the user at the time of image capture. Also, informationstored such as clothing sizes, either in the CI device or in a remotedatabase, enables the system to extract only the locations having thecorrect items in stock.

The personal profile might also include the user's asset information,facilitating payments and/or refunds. There has been recent speculationthat cellular telephones will assume functionality of credit cards,identification means, access means, and the like. This functionalitycertainly is adaptable to the presently disclosed system.

Advertising is another commercial application of the present invention.For example, a user captures an image of an advertisement in a magazine,on a poster, or on the screen of a television, transmits the advertisingimage to a database, and acquires additional information about theproduct, commodity, or service. The user may also be linked to thesource site of the advertisement. Advertisements might be captured frompamphlets, flyers, newspapers, books, posters, magazines, newspapers, TVcommercials, coupons, or the like. Alternatively, information aboutservices involving matters of health, law, travel, insurance, and thelike also may be acquired. For example, a person can “shoot” a movieposter or marquee to obtain reviews of a movie, times and places ofshowing, cost of tickets, information about the director and actors, andinformation about other movies that might appeal to the user. The userof the system can also purchase tickets.

FIG. 10 shows an example of the aforementioned application. Anadvertisement in magazine 1000 is scanned into computer 1002 throughscanner 1001, wherein scanner 1001 and computer 1002 comprise elementsof a CI device 1010. The user highlights the particular advertisement1005 of the magazine page for which additional information is desired.Computer 1002 sends the image to database 1004, where the image isprocessed and compared with images stored in the database. The systemextracts information 1006 about advertisement 1005, such as a moredetailed description. The system can also provide a link to the sourcesite of information 1007 or directly take the user to the source site asthe advertisement is entered into the search engine. This aids the userin finding contact or pricing information to purchase services orproducts. The system also offers a listing of related sites 1008 wherethe user might access similar categorized services.

FIG. 11 provides an example of another application of the presentinvention involving communication with or acquisition of informationabout persons of interest to the user. Upon observing person of interest1100, the user of the system uses CI device 1101, such as a cellulartelephone having a camera, to acquire information about person ofinterest 1100 by capturing their image 1102. CI device 1101 sends theimage to database 1105 by means of wireless transmitting device 1103 forprocessing of that image. The processing means associated with database1105 performs facial recognition or some other form of biometricrecognition, identifies the individual, and extracts informationregarding the individual from the database 1105. The user of the systemcan then view the information on the display screen of CI device 1101.

Database 1105 contains images linked to information regarding theindividual in the image. Upon extracting information, the system sends aweb link to CI device 1101, or it downloads the information onto CIdevice 1101. The individual's information can be as extensive as theindividual chooses, depending on system applications such as personalinterests, professional interest, medical history, criminal history,commercial preferences, or other similar information. The informationcan be entered in profile form 1110 by the person of interest and canconsist of the individual's name, screen name, description, textinformation, visual features, personal traits, demographiccharacteristics, additional photographs, audio clips, video clips, orthe like. However, due to security and/or privacy issues, after the usercaptures an image and desires to extract further information about theperson of interest, the system notifies the person of interest orrequests permission to allow the user of the system to access theperson-of-interest's data. Additionally, the person of interest mightfirst request information from the user, such as photographs, maritalstatus, educational background, professional status, level of income,ethnicity, political beliefs, and the like, before sending or permittingthe access of any personal information. As well, any person may choosenot to publicly post information to ensure his or her privacy. In thiscase, the user of the system is unable to extract any information.

On the other hand, if the person of interest seeks to find people withcommon interests, his or her profile 1110 may contain contactinformation that enables the inquiring user of the system to establishcontact. The contact information can be a phone number, an address, ane-mail address, an instant messenger screen name, or an anonymouscontact capability. For example, if a instant messenger screen name isavailable, the user uses CI device 1101 to send instant message 1111 toperson of interest 1100 through the Internet, and person of interest1100 decides whether or not to respond. For security purposes, thesystem may request that the user first transmit his or her profile tothe person of interest, providing a basis for the person of interest todecide whether or not to maintain or expand contact. The user of thesystem may also use CI device 1101 to capture an image of multipleindividuals in a facility. The system may provide the user with aselective choosing device, such as a scroll button, a mouse, or anumbering system, to select persons of interest and to acquire theirinformation.

The system need not be used only in the Internet context. Variousorganizations can use the system to identify people or acquire importantinformation. The database could be maintained by the organizations andcontain data such as the image representation of an individual and theirdesired descriptions. In the medical field instance the database can bemaintained by medical facilities and entered by medical personals as theindividuals' medical records change. For example, individuals having achronic illness may contain information identifying their illness andways of assisting such an individual. In the instance that they have arecurrence, anyone authorized to access the system, such as medicalpersonnel, may capture their image and acquire password protectedmedical information about the affected person through available wirelessInternet based device. This would provide a more secure environment forthe sickly, elderly, or the like.

The system can also be used in a secure environment such as, but notlimited to a prison, airport, secret agencies, army, hospital, and thelike. In these applications, the individual's information includescriminal, immigration, medical records, or the like. Anyone that hasaccess to the system can enter the information about the person ofinterest into the system. The authorized person can access this personalinformation via a password or the like. The information can be used torun background checks, to identify individuals in need of help, to findmissing individuals, or the like.

The system can be used as a child-loss-prevention system wherein parentsor school officials may enter the child's photograph along withidentifying information. The identifying information may include thechild's name, names and contact information of parents or schoolofficials, and/or the address of the family's residence. For instance,if a user of the system finds a lost child, that user can use the CIdevice to capture a photograph of the child and acquire the identifyinginformation regarding the child.

The present invention also has great potential for the field ofeducation. The present invention provides a system and method foraccessing information regarding an object of question. The source ofinformation retrieved can be books, dictionaries, encyclopedias,articles, news, or the like. FIG. 12 illustrates a means for accessingeducational information in which a user of the system captures an imageof puppy 1200 and enters it into computer 1202 through the computer'sconnection with video camera 1201 that is associated with a CI device1210. The image is entered into search engine 1204 through network 1203and is processed in order for best-fit matches to be found in database1204. A listing of possibilities may then be transmitted back to the CIdevice 1210 and displayed on display unit 1205 or 1211 associated withthe CI device 1210.

The present invention has further educational potential. For example,the user can capture images of exhibited art or artifacts and enter theimages into the image search engine to acquire historic or otherinformation.

An example of a virtual travel guide is shown in FIG. 13. It utilizesGPS and an Internet-accessible digital camera 1300 as the CI device. TheCI device captures image 1301 of building 1302 and acquires information.The CI device 1300 can also acquire the time and the date of the imagecaptured, and GPS information from satellite 1304 through wirelesstransmitting device 1303 and transfer the captured information todatabase 1306 through network 1305 as a data packet. The GPS coordinatesare used as a search constraint to refine the search. When populatingsuch a database, it is desirable to associate different scenarios withimages as well as location coordinates. When a data packet received bydatabase 1306 contains both images and coordinates, the database firstsearches for the closest matching coordinates until a specific range isreached. The captured image is then checked against stored images whosecoordinates fall within the specified range. As a result GPS narrows thesearch and thereby expedites the extraction of useful information.

The acquired information is displayed on display screen 1314 of CIdevice 1300. If captured image 1301 is of a historic building, theinformation can include the name of the building and a historic profile1310, including the building's dimensions, the building establishmentdate, the past usage of the building, and the like. The user of thesystem enters an inquiry date into the CI device in order to extractinformation associated with the image on that particular date orthereabouts. The information is extracted from a preset timeline ofevents, or is used as a search entity to search the Internet.Additionally, date and time 1311 of the captured image is used toextract information associated with that time and the date. For example,the time and date might be associated with information as to whether ornot the building is open to visitors. The current time and date coned tobe used to extract current news involving building 1312. The databasealso might search the Internet for available new information posted onthat specific time and date and display sites 1313 on the CI device.

Additionally, the system may be used as a translation or dictionaryguide to translate signs or written documents. For example, the systemcaptures an image of a street sign in a language foreign to the user andfurther uses GPS coordinates for assistance to determine the country inwhich the image is captured. The system further performs imageprocessing to identify the written characters and input the written orprinted word or phrase into an electronic translator.

1. A method comprising: storing information regarding a plurality of images of advertisements and information regarding a plurality of products, commodities, or services in a database, wherein at least one image of an advertisement is associated with a product, commodity, or service of the plurality of products, commodities, or services; receiving, with a receiver, information regarding an image of an advertisement of a product, commodity, or service of interest, wherein the image of the advertisement of the product, commodity, or service of interest is obtained with a communication device of or associated with a requesting user, and further wherein the communication device transmits the information regarding the image of the advertisement of the product, commodity, or service of interest to the receiver; comparing the information regarding the image of the advertisement of the product, commodity, or service of interest to the information regarding the plurality of images of advertisements in the database; determining whether the image of the advertisement of the product, commodity, or service of interest matches an image of the plurality of images of advertisements, and, if the image of the advertisement of the product, commodity, or service of interest matches an image of the plurality of images of advertisements, accessing information regarding the associated product, commodity, or service; and transmitting, using a transmitter, the information regarding the associated product, commodity, or service to the communication device of or associated with the requesting user; wherein the information regarding the associated product, commodity, or service is displayed on a display unit of or associated with the communication device.
 2. The method of claim 1, further comprising: obtaining the image of the advertisement of the product, commodity, or service of interest from a printed material, wherein said printed material is selected from the group consisting of a magazine, a newspaper, a poster, a pamphlet, a flyer, a book, an identification card, a credit card, a bank card, a passport, a photograph, and any combination thereof.
 3. The method of claim 1, further comprising: obtaining the image of the advertisement of the product, commodity, or service of interest from an electronic display unit.
 4. The method of claim 3, wherein said electronic display unit is selected from the group consisting of a television screen or monitor, a computer monitor, a screen of a hand-held device, and any combination thereof.
 5. The method of claim 4, wherein the advertisement of interest displayed on said television screen or monitor comprises a television commercial.
 6. The method of claim 1, wherein the information regarding the associated product, commodity, or service comprises at least one selected from the group consisting of additional information about the associated product, commodity, or service being advertised; a more detailed description of the product, commodity, or service; pricing information; discounts; store locations; availability; online purchase capabilities; purchase statistics; information about related products, commodities, or services; a link to retail store site; a link to a product manufacturer site; a link to an online auction site; and any combination thereof.
 7. The method of claim 1, further comprising: providing to the requesting user the capability to take a subsequent action selected from a group consisting of making a purchase of a product, commodity, or service; leasing a product; arranging financing; arranging delivery of a product, commodity, or service; arranging pick-up of a product or commodity; and any combination thereof.
 8. The method of claim 1, wherein the information regarding the associated product, commodity, or service is displayed based on criteria stored in a profile of the requesting user.
 9. The method of claim 8, wherein the user profile is stored in the communication device or in the database.
 10. The method of claim 8, wherein the criteria in the user profile comprise at least one selected from the group consisting of price limits, residency, taste, sizes, place of residence, workplaces, current location, asset information, and any combination thereof.
 11. The method of claim 1, wherein the requesting user captures the image of the advertisement of a product, commodity, or service of interest and then accesses additional information about availability of variations of the product, commodity, or service and locations to purchase it.
 12. The method of claim 1, wherein the communication device is selected from the group consisting of a hand-held device, cellular telephone, hybrid cellular telephone/PDA device, PDA, remote server, RFID device, Internet-accessible camera, personal computer, laptop computer, pocket computer, hybrid electronic device, and any combination thereof.
 13. The method of claim 1, wherein the communication device comprises at least one selected from the group consisting of a camera, a scanner, a processor, a display, or a combination thereof.
 14. The method of claim 1, further comprising: processing information, via the communication device, for performing image recognition pertaining to the information regarding the image of the advertisement of the product, commodity, or service of interest.
 15. The method of claim 1, further comprising: processing information, via a processor, for performing image recognition pertaining to the information regarding the product, commodity, or service of interest.
 16. The method of claim 1, further comprising: accessing the apparatus through a network.
 17. The method of claim 16, wherein the network comprises at least one selected from the group consisting of a wired network, a wireless network, a GSM network, a ISDN network, an Ethernet network, a CATV network, a Wi-Fi network, a LAN network, a Bluetooth network, and any combination thereof.
 18. A system for enabling advertisers or marketers to preplan response to the entry of images of advertisements into a search engine by providing images of said advertisements to those who update the search engine, comprising: a database storing information regarding a plurality of images of advertisements and information regarding a plurality of products, commodities, or services, wherein at least one image of an advertisement is associated with a product, commodity, or service of the plurality of products, commodities, or services; a receiver receiving information regarding an image of an advertisement of a product, commodity, or service of interest, wherein the image of the advertisement of the product, commodity, or service of interest is obtained with a communication device of or associated with a requesting user, and further wherein the communication device transmits the information regarding the image of the advertisement of the product, commodity, or service of interest to the receiver; a processer for comparing the information regarding the image of said advertisement of the product, commodity, or service of interest to the information regarding the plurality of images of advertisements, determining whether the image of the advertisement of the product, commodity, or service of interest matches an image of the plurality of images of advertisements, and, if the image of the advertisement of the product, commodity, or service of interest matches an image of the plurality of images of advertisements, accessing information regarding the associated product, commodity, or service; a transmitter transmitting the information regarding the associated product, commodity, or service to the communication device of or associated with the requesting user; and a display for displaying the information regarding the associated product, commodity, or service.
 19. The system of claim 18, wherein said advertiser or marketer provides links to additional information regarding the product, commodity, or service of interest.
 20. The system of claim 18, wherein said additional information is selected from the group consisting of a more detailed description of the product, commodity, or service, pricing information, discounts, store locations, availability, online purchase capabilities, purchase statistics, information about related products or services, and any combination thereof.
 21. The system of claim 18, wherein the communication device provides to the requesting user the capability to take a subsequent action selected from the group consisting of making a purchase of a product, commodity, or service, leasing a product, arranging financing, arranging delivery of a product, arranging pick-up of a product, and any combination thereof. 